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USE  OF  SYNTHETIC  BENCHMARKS  FOR  ESTIMATING 
SERVICE  BUREAU  PROCESSING  CHARGES 

by 

Dennis  M.  Conti 


ABSTRACT 


This  report  describes  the  development  of  a  new  synthetic 
benchmark  technique  for  estimating  batch  processing  charges 
at  service  bureau  sites.   This  technique  was  used  to  esti- 
mate the  cost  of  processing  a  large  batch  workload  at  a 
number  of  service  bureaus  within  the  same  mainframe  family. 
The  method  was  found  to  be  low-cost,  yet  reasonably  accurate 
for  a  certain  class  of  service  bureau  charging  algorithms. 
Refinements  of  this  method  are  suggested  which  will  extend 
its  applicability  to  other  algorithms.   The  procedures  used 
to  create  and  run  the  benchmark,  together  with  the  projection 
of  total  workload  cost  are  described. 

Key  Words:   Benchmarking;  charging  algorithms;  service 
bureaus;  synthetic  benchmarking;  workload 
characterization. 


I.   INTRODUCTION 

Federal  agencies  with  a  requirement  for  ADP  products  and  services 
and  subject  to  Office  of  Management  and  Budget  Circular  A-76,  "Policies 
for  Acquiring  Commercial  or  Industrial  Products  and  Services  for 
Government  Use,"[l],  may  find  it  necessary  to  compare  the  cost  of  data 
processing  at  commercial  service  bureaus  and  the  cost  of  doing  the  same 
processing  in-house  at  Federally-owned  and  operated  facilities.   This, 
in  turn,  has  created  a  requirement  for  a  low-cost,  reasonably  accurate 
method  which  agencies  can  use  to  project  their  processing  costs  at 
various  service  bureau  sites.   This  report  describes  preliminary  work 
undertaken  by  the  National  Bureau  of  Standards  to  develop  such  a 
method. 

For  input  to  an  A-76  cost  study  it  was  undertaking,  the  Data 
Management  Center  (DMC)  of  the  Department  of  Health,  Education,  and 


Welfare  requested  that  the  Institute  for  Computer  Sciences  and 
Technology  of  the  National  Bureau  of  Standards  (NBS/ICST)  assist  it  in 
estimating  the  cost  to  process  its  batch  workload  at  commercial  service 
bureau  sites.   The  scope  of  the  project  was  to  include  only  direct  batch 
processing  charges — no  attempt  would  be  made  to  estimate  teleprocessing 
or  other  ancillary  charges  (e.g.,  storage  charges  for  tapes,  terminal 
connect  charges,  etc.).   Furthermore,  the  study  was  to  consider  only 
service  bureaus  with  DMC- compatible  systems  (IBM  OS  and  VS  systems  in 
the  370/155-370/168  range).   Two  approaches  were  investigated,  as 
described  below. 

The  charging  algorithms  of  a  number  of  service  bureaus  were  to  be 
obtained  by  NBS/ICST  and  coded  into  a  COBOL  program.   The  system 
accounting  log  data  (SMF  [2])  for  a  representative  month  was  to  be 
processed  by  this  program  and  an  estimate  of  service  bureau  charges, 
based  on  their  respective  algorithms,  was  to  be  obtained.   This  method 
did  in  fact  produce  very  accurate,  job-by-job  cost  projections  for  each 
service  bureau  [3].   Two  difficulties  with  this  method,  however,  were: 
(1)  the  need  to  keep  the  highly-sensitive  service  bureau  algorithms 
confidential;  and  (2)  the  difficulty  in  verifying  that  the  COBOL  version 
of  the  algorithms,  and  indeed  the  version  supplied  by  the  service 
bureaus,  were  valid. 

The  second  approach  was  to  develop  a  method  by  which  the  DMC's 
batch  workload  could  be  represented  by  a  small  set  of  synthetic  benchmark 
programs.   The  programs  would  be  run  at  the  same  set  of  service  bureaus 
selected  for  the  algorithm  approach.   The  total  DMC  workload  charges 
would  then  be  projected  from  the  actual  charges  incurred  by  the  bench- 
mark jobs.   The  purpose  of  the  synthetic  benchmark  approach  was  not  only 
to  provide  a  validity  check  against  the  algorithm  approach,  but  also  to 
determine  if  a  low-cost,  yet  reasonably  accurate  method  for  estimating 
batch  processing  charges  could  be  developed. 

The  objective  of  this  report  is  to  describe  the  synthetic  benchmark 
approach  taken  to  estimate  the  cost  of  processing  DMC's  workload.   Other 
agencies  may  find  these  techniques,  or  extensions  thereof,  useful  in 
their  own  procurement  environment. 


This  report  begins  with  a  description  of  previous  attempts  to 
model  batch  workloads  with  synthetic  programs  (primarily  for  the  purpose 
of  thruput  estimation),  and  the  relationship  of  this  past  work  to  the 
approach  taken  here.   Two  initial  attempts  to  construct  an  acceptable 
synthetic  mix  are  then  described.   A  detailed  discussion  then- follows 
of  the  assumptions  and  theory  underlying  the  approach  ultimately  taken. 
Finally,  the  procedures  used  to  run  the  synthetic  programs  and  to 
project  the  total  workload  processing  costs  are  described. 


II .   BACKGROUND 

Synthetic  benchmarks  can  be  classified  into  two  categories:   task- 
oriented  and  resource-oriented.   Task-oriented  synthetics  are  artificial 
programs  which  attempt  to  model  the  functions  of  the  real  workload, 
while  resource-oriented  synthetics  attempt  to  model  the  resource  demands 
which  the  real  workload  places  on  a  system.   Because  of  the  difficulty 
in  obtaining  functional  descriptions  for  most  jobs  in  a  workload, 
resource-oriented  synthetics  have  been  investigated  more  extensively. 

Historically,  resource-oriented  synthetics  have  been  used  to  tune 
existing  systems  or  to  benchmark  new  systems.   In  1969  Buchholz  [4] 
described  a  parameter-driven  PL/I  program  which  he  claimed  could  be 
used  as  "a  well-behaved  exerciser  of  system  features  or  a  tool  for  com- 
paring the  speed  of  dissimilar  systems."  The  Buchholz  program  was  a 
generalized  file  maintenance  program  which  operated  on  two  ordered  files , 
a  master  file  and  a  detail  file.   The  program  would  sequentially  read  the 
master  file  until  a  record  was  found  which  matched  the  current  detail 
record.   Upon  detection  of  a  match,  a  compute-bound  kernel  would  be 
executed.   An  attempt  would  then  be  made  to  match  the  next  detail  record. 
This  process  continued  until  the  end  of  the  master  file  was  reached. 
Input  parameters  could  be  chosen  to  control  the  number  of  master  and 
detail  records,  and  the  number  of  times  the  compute  kernel  was  executed 
following  a  master-detail  match.   The  Buchholz  program  has  been  used  as 
the  basis  for  many  resource-oriented  synthetic  benchmarks. 


Crowding  [5J  attempted  to  represent  a  real  workload  by:   (1) 
categorizing  user  jobs  into  core  classes;  (2)  computing  the  "average 
job"  for  each  class;  and  (3)  constructing  a  resource-oriented  synthetic 
job  to  represent  each  class.   These  synthetic  jobs  were  then  combined  to 
represent  the  total  workload.   Crowding  was  able  to  obtain  close  agree- 
ment between  machine  utilization  data  for  the  real  workload  and  that  for 
the  synthetic  job  stream  on  a  single  system.   Kernighan  and  Hamilton  [6] 
also  achieved  success  when  they  attempted  to  represent  a  workload  by  a 
set  of  synthetic  jobs  which  exercised  only  the  CPU,  10,  and  core. 
Furthermore,  they  felt  that  "moving  [their]  benchmarks  from  one  machine 
or  system  to  another  is  simple"  and  that  their  approach  was  "quite 
amenable  to  comparisons  of  system  1  on  machine  A  with  system  2  on 
machine  B." 

One  of  the  latest  and  more  sophisticated  uses  of  synthetic  benchmarks 
is  described  by  Sreenivasan  and  Kleinman  [7],   They  attempted  to  repre- 
sent the  real  workload  by  a  joint  probability  matrix  of  CPU  time  and 
10  counts.   That  is,  for  each  (CPU,  10)  pair  the  probability  was  calcu- 
lated that  a  job  existed  in  the  real  workload  with  processing  time 
'CPU'  and  EXCP1  count  '10'.   A  "drive  workload"  consisting  of  a  Fortran 
version  of  the  Buchholz  synthetic  was  constructed  to  match  this  CPU-IO 
distribution.   Since  most  of  the  jobs  in  the  real  workload  required  a 
128K  byte  partition,  all  of  the  synthetic  jobs  were  forced  to  run  in 
128K  bytes.   Sreenivasan  and  Esposito  [8]  later  extended  this  work  in  a 
study  of  a  Burroughs  system  to  include  core  as  a  third  variable  to  be 
modeled.   They  concluded  that  using  their  approach,  "the  relative 
performance  of  dissimilar  systems  can  be  evaluated."   King  [9]  describes 
an  automated  procedure  which  models  the  real  workload  by  matching 
the  resource  requirements  of  a  set  of  synthetic  jobs  with  the  resource 
requirements  of  user  jobs  in  the  real  workload.   He  concludes  that 
this  automated  procedure  solves,  among  other  things,  the  "transportation 
problem"  of  benchmark  jobs  to  a  "strange  environment." 

These  previous  attempts  to  model  a  real  workload  by  synthetic  jobs 
had  as  their  primary  goal  the  evaluation  of  system  performance. 


EXCP  -  a  measure  of  the  number  of  10  block  transfers  on  IBM  systems. 


Furthermore,  several  of  these  authors  have  suggested  that  resource- 
oriented  synthetics  can  be  used  in  the  comparison  of  dissimilar  systems. 
The  use  of  resource-oriented  synthetics  for  this  purpose  has  been  one  of 
the  most  controversial  issues  of  synthetic  benchmarking  which  others 
have  questioned  [10,11].  While  the  approach  used  in  this  study  employed 
resource-oriented  synthetics  to  model  the  DMC  workload,  its  use  was 
restricted  to  estimating  processing  costs  within  a  single  family  of 
systems.   These  were  two  distinct  differences  from  the  previous  approaches 
which  used  synthetics  to  place  a  representative  load  on  a  system  (or  on 
systems  across  vendors)  in  order  to  obtain  thruput  and  turnaround  times. 
In  the  study  described  here,  resource-oriented  synthetics  are  used  only 
to  drive  charging  algorithms  in  a  representative  manner. 


III.   PRELIMINARY  EFFORTS 

The  first  two  attempts  to  construct  a  synthetic  benchmark  represen- 
tative of  DMC's  workload  proved  unsatisfactory  for  a  number  of  reasons. 
Either  the  cost  to  run  the  benchmark  at  each  service  bureau  would  have 
been  excessive,  or  else  the  benchmark  would  not  accurately  represent  the 
DMC  workload.   However,  a  detailed  analysis  of  the  general  nature  of  the 
service  bureau  algorithms  provided  the  insight  that  allowed  for  the 
construction  of  a  reasonably  accurate,  low-cost  synthetic  benchmark. 
This  section  first  describes  the  required  SMF  data  preparation  steps 
and  then  discusses  the  initial  attempts  to  produce  an  acceptable 
synthetic  mix. 

A.   SMF  Data  Preparation 

A  major  assumption  of  this  study  was  that  the  workload  characteristics 
captured  by  SMF  were  typical  of  the  DMC  workload  and  could  serve  as  the 
primary  basis  for  dollar  charges  at  commercial  service  bureaus.   Before 
application  of  either  the  algorithm  or  synthetic  approach  could  commence, 
therefore,  a  data  base  consisting  of  DMC's  SMF  records  for  the  month  of 
July  1974  (chosen  by  DMC  as  a  representative  month),  grouped  by  job,  was 


required.   In  addition,  certain  anomalous  cases  (e.g.,  step  records  with 
no  corresponding  job  termination  record)  needed  to  be  identified,  and 
all  improper  records  eliminated.   The  following  describes  the  sequence 
of  steps  taken  to  construct  this  final  data  base. 

SMF  tapes  were  provided  at  the  outset  of  the  study  which  contained 
all  SMF  record-types  collected  at  DMC  during  the  month  of  July  1974.   In 
addition,  all  account  numbers  and  job  names  were  suitably  encoded. 
Because  they  were  the  only  records  of  interest  in  this  study,  the 
following  two  record-types  were  extracted  from  these  tapes: 

Type  4  -  step  termination  record  -  89,700 

Type  5  -  job  termination  record  -  26,352 
See  the  IBM  SMF  manual  [2]  for  a  complete  description  of  each  record 
type. 

The  SMFCOPY  program  (a  utility  provided  by  DMC)  was  used  to  extract 
the  above  records  for  batch  jobs  only,  thereby  reducing  the  number  of 
records  to  the  following: 

Type  4  -  89,084 
Type  5  -  25,850 
This  intermediate  file  was  sorted  first  by  job-log- number  (i.e.,  job 
name,  reader  start  time,  reader  start  date),  and  then  by  record  time- 
stamp  within  each  job.   Because  of  the  volume  of  data,  two  sorts  and 
then  a  merge  were  actually  required. 

Finally,  this  set  of  data  was  examined  for  consistency  (every  type 
4  had  a  corresponding  type  5,  every  type  5  had  at  least  one  4),  and 
extraneous  records  (type  4  primarily)  were  deleted.   This  resulted  in  a 
final  data  base  containing  86,392  step  termination  and  25,850  job 
termination  records. 

B.   Initial  Attempts 

Before  describing  the  synthetic  approach  actually  used  in  this  study, 

two  attempts  to  produce  a  representative,  yet  low-cost  synthetic  mix 

will  now  be  discussed. 

The  first  such  attempt  consisted  of  the  following  steps: 

1.   The  type  4  records  were  partitioned  into  the  three  highest 


revenue-producing  job  classes:   H,  D,  and  J.   A  fourth  class, 
OTHER,  included  the  remaining  job-step  records. 

2.  For  each  job  class,  the  job-step  records  were  divided  into  core 
groups  of  40K  increments. 

3.  For  each  core  group,  a  CPU-IO  joint  probability  matrix  was 
computed. 

4.  A  synthetic  mix  of  N  jobs  was  then  constructed  whose  joint 
probability  matrix  matched  that  of  the  real  workload,  but  whose 
total  CPU  time  did  not  exceed  one  hour. 

5.  This  mix  of  N  jobs  was  then  to  be  run  at  the  selected  service 
bureaus.   Appropriate  expansion  factors  were  to  be  applied  to 
the  individual  job  costs  to  determine  total  projected  workload 
costs. 

The  above  approach  closely  paralleled  the  synthetic  benchmark  generation 
process  outlined  by  Sreenivasan  and  Kleinman  [7].   Several  difficulties 
were  encountered  with  this  approach,  however.   First,  only  the  dominant 
(CPU,  10)  pairs  could  be  represented  in  the  final  synthetic  mix  because 
of  the  need  to  satisfy  the  one  hour  total  CPU  time  criterion.   Second, 
even  with  this  final  mix,  minimum  job  charges  at  several  service  bureau 
sites  would  have  overly  biased  the  resulting  projected  workload  costs. 
A  modified  version  of  this  approach  was  then  tried. 

Each  (CPU,  10,  core)  combination  was  first  ranked  according  to  total 
resource  usage  (i.e.,  the  sum  of  CPU  time,  EXCP  counts,  and  allocated 
core).   The  relative  contribution  of  each  of  the  four  job  classes  to 
total  workload  CPU  time  was  calculated.   The  proportionate  CPU  contri- 
bution of  each  class  to  a  total  mix  time  of  one  hour  was  then  computed. 
Figure  1  depicts  these  results. 


Real  Workload 

Class 

CPU  Time  (sec) 

% 

of  Total 

H 

192,114 

31.2 

D 

71,338 

11.6 

J 

100,901 

16.4 

OTHER 

251,638 
615,991 

40.9 

Synthetic  Mix 
Allotted  CPU  Time  (sec) 

562 

209 

295 

736 


1802  sec  (  1  hr) 


Figure  1 


For  each  job  class,  the  top-ranked  (CPU,  10,  core)  groups  were  selected 
until  cumulative  CPU  time  exceeded  the  allotted  time  for  that  job  class. 
A  synthetic  job  was  then  constructed  to  represent  each  of  the  selected 
groups.   The  collection  of  synthetic  jobs  from  all  job  classes  consti- 
tuted the  benchmark  mix.   This  mix  was  actually  run  at  two  service 
bureaus.   However,  because  of  the  cost  incurred  for  these  first  two  runs, 
and  because  it  was  felt  that  this  mix  did  not  adequately  represent  the 
real  workload  (since  many  resource  groups  were  not  represented) ,  this 
approach  was  abandoned. 

It  then  became  clear  that  with  a  few  reasonable  assumptions 
concerning  the  nature  of  the  service  bureau  algorithms,  a  more  accurate, 
low-cost  approach  could  be  developed.   This  approach  proved  to  be  suc- 
cessful for  a  number  of  service  bureau  algorithms  and  is  now  described 
in  more  detail. 


IV.   SELECTED  APPROACH 

A.   Assumptions 

The  main  assumptions  specific  to  the  approach  taken  in  this  study 
are  the  following: 

1.   CPU  time,  10  channel  activity,  and  core  memory  space  are  the 
major  cost  factors  for  a  job — all  other  miscellaneous  charges 
(MISC)  have,  a  secondary  effect  on  total  job  charges. 


2.  Service  bureau  charging  algorithms  are  of  the  general  form: 

$  =  W  -CPU  +  W„-IO  +  W„ -CPU-Core  +  W  -IO-Core  +  W  -MISC. 
L  Z  3  4  5 

3.  The  assigning  of  jobs  to  discrete  core  breakpoints  has  only  a 
minor  effect  on  total  job  charges. 

4.  A  synthetic  job  can  be  constructed  to  place  a  prescribed  load 
on  the  CPU,  10  channels,  and  core. 

Each  of  these  assumptions  will  be  discussed  in  more  detail  in  the 
following  paragraphs. 

B.   Theory 

The  following  describes  in  theoretical  terms  the  selected  synthetic 

benchmark  process  and  its  adequacy  as  a  tool  for  comparing  the  cost  of 

processing  a  given  workload  by  different  service  bureaus.   Let 

R  =  {R-,...,R  }  be  the  set  of  resources  to  be  modeled.   Because  core, 
1      m 

the  CPU,  and  10  channels  are  assumed  to  be  primary  contributors  to  a 
job's  cost,  they  were  chosen  as  the  set  R. 

A  synthetic  job,  S,  is  a  program  capable  of  driving  each  of  the  m 
resources  in  a  prescribed  manner.   The  demand  which  S  puts  on  each  of 
the  resources  is  a  function  of  specific  values  of  the  input  parameter 
list  P  =  (P..,...,P  ).   In  order  that  this  functional  relationship  may 
be  empirically  determined,  S  is  first  calibrated  on  the  base  machine 
over  a  wide  range  of  values  for  each  P..   A  linear  regression  model  or 
source  code  analysis  is  then  used  to  determine  some  function,  f, 
between  P  and  the  utilization  of  each  resource  in  set  R.   For  example, 
assume  that  a  relationship  is  found  between  the  CPU  time  and  EXCP  count 
for  S  and  the  parameter  list  P: 

(CPU,  EXCP)  =  f (P). 

Then,  given  a  prescribed  CPU  time,  t,  and  a  desired  EXCP  count,  d,  the 
program  S  with  parameter  list  P  =  f~  (t,d)  should  execute  in  CPU  time  t 
and  should  produce  d  EXCP's. 


Assume  that  all  jobs  in  the  real  workload  are  partitioned  into  r 
core  groups : 


C-.  j  C  , . . .  ,C 


where 


c.  =  a,  .  +  a 

i       1-1 

is  the  i'th  core  breakpoint.   For  this  study,  A  =  40K  bytes.   A  job  is 
assigned  to  core  group  C.  if  its  core  size  falls  within  i  t  of  C. .   Let 
CPU. .  and  10. .  be  the  CPU  time  and  10  channel  activity  (expressed  in 
EXCP  counts)  for  the  j  '  th  job  of  core  group  C. .   For  each  C. ,  let: 


E 

CPU.  . 
13 

CPU. 

1 

_  j 

n. 

i 

E 
=  J 

10.  . 

ij 

10. 

X 

n. 

represent  the  CPU  time  and  EXCP  counts,  respectively,  for  the  "average 
job,"  where  n.  is  the  number  of  jobs  in  C .  .   Let  sf.  be  the  appropriate 
scaling  factor  which  reduces  CPU.  to  an  arbitrary  value,  t'  (here,  t' 
was  chosen  to  be  10  seconds) : 


1  if  CPU.  <  t* 
l  — 

sf.  = 

1  CPU. 

r^  if  CPU.  >  t' 

t'        l 
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Calculate  the  new  scaled  averages : 


'       1 
cpu.  =  -4- 

x        sf . 

X 

•    CPU. 

X 

k>:-  \ 

x        sf . 

•  io. 

X 

-1   •    ' 

Finally,  determine  the  appropriate  input  parameter  list  p.  =  f   (CPU.,  10.) 

X  •  XX 

*         1 

which  causes  S  to  exactly  duplicate  the  scaled  averages  CPU.  and  10. . 
If  S.  represents  S  forced  to  a  core  size  of  C.  (e.g.,  via  the  REGION 
parameter  on  the  EXEC  card),  then  S.(p.)  will  be  used  to  represent  all 
jobs  in  core  group  C. . 

Inspection  of  the  actual  service  bureau  charging  algorithms  supports 
the  claim  that  many  are  of  the  general  form: 


$  =  W  -CPU  +  W  -10  +  W  • CPU -Core  +  W  -IO-Core  +  W  -MISC 

where  MISC  are  miscellaneous  charges  that  are  assumed  to  have  a  secondary 

effect  on  the  total  cost  of  a  job  (e.g.,  disk  and  tape  allocation 

charges,  charges  for  temporary  work  space).   Note,  however,  that  the 

following  implicit  assumptions  still  remain:   1)  no  minimum  job  charges 

exist  in  any  of  the  service  bureau  pricing  schemes;  and  2)  disk  and  tape 

channel  activity  have  the  same  weighting  factor  (i.e.,  W, ) .   The  cost  of 

actually  running  a  job  J.  .  with  core  size  C.  +6..  (|S..  <  tt)  ,  CPU  time 
6   J    xj  x    xj  V|  xj  '-  2 

CPU..,  EXCP  count  10..,  and  miscellaneous  charges  MISC..,  is  thus: 

$. .  =  W  -CPU. .  +  W  -10.  .  +  W  -CPU.  . • (C.  +  6. .)  +  W, -10. . -(C.  +  6.  .) 
ij     1    ij     2   xj     3    xj    x    xj'     4    xj    x    xj 

+  W  -MISC. . 
5     ij 
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The  actual  cost  of  running  all  jobs  represented  by  core  group  C.  is 
then: 


(1)  $actual.  =  E  $..  =  W  -E  CPU..  +  W  -E  10. 

1    .   ij    1      in    2  .    it 
3  3  3    J 


+  W  -C.-E  CPU. . 
3   l  .     it 
3 


+  W  *E  CPU. .-6..  +W  -C.-E  10.. 
3  .     ii   ii     4   l  .    ii 
3  3 


+  W.  •  E  10 .  .  •  6  .  .  +  W  •  E  MISC .  . 


and  the  actual  cost  of  all  jobs  in  all  core  groups  is: 


$actual  =  E  $actual. 

l 

l 


Now,  assume  that  the  synthetic  job  S.(p.)  is  run  at  the  service 


bureau.   Its  cost  will  be: 


$ (S . )    =  W    • CPU .    +  W    • 10 .    +  W    • CPU . • C .    +  W. • 10 . • C .    +  W   • MISC . 
l  1i2i3ii4ii5  i 

where  MISC.  is  primarily  due  to  the  synthetic's  temporary  work  space. 


Expanding  the  above,  we  obtain: 
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1  1 


1  1  ' 

+  W   •-=-*  CPU.-C.    +W  .  — 4— 10   -C.    +  W    -MISC. 
3  sf.  11  A   sf.        l     i  5  i 

l  l 


E   CPU. .               E   10. 
=  W,  ,j-t +  W    -J 


1   sf  .  -n.  2   sf .  'Xi. 

11  li 


£   CPU . .  E   10.. 

+  W    •  C .  -^= +  W    •  C .  -3-z +  Wc  •  MISC  . 

3     l  sf.*n.  4     l   sf.-n.  5  i 

li  li 


Let  ef.  =  sf.'n.  be  the  expansion  factor  for  S..   The  estimated  cost  for 
ill  l 

all  the  jobs  in  core  group  C.  is  then: 


(2)  $est.  =  ef.-$(S.)  =  W  -Z  CPU.. 

l     11     l.ii 


+  W  •£  10. . 

2  i    1J 


+  w  *c. -s  CPU. . 
3  i  .    ij 
3 


+  W  -C.-S  10. . 
4  l  .    ij 
J 


+  W  -sf . -n. -MISC. 
5   l  i     i 


Comparing  (1)  and  (2),  this  estimate  is  in  error  by: 
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e.    =   $actual.    -   $est.    =  W  •  E   CPU.. -6..   +  W   -Z   10      -6 
i  i  i  3    .  ij      ij  4    .        ij      ij 

■J  J 


+  W    •  (Z   MISC..    -   sf.-n.-MISCj 
5      .  U 


The  estimated  cost  of  the  complete  workload  W  is  given  by 


$est  =  Z  $est. 
l 

i 


and  is  in  error  by: 


e  =  $actual  -  $est  =  Z  e. 

i 


=  W  •£  CPU. . -6. .  +  W  -Z  10. . -6. . 

3  .  .    ii   ij    4  ,.   ij   ii 


+  W  -(Z  MISC. .  -  MISC  ) 
ij    XJ 


where  MISC  =  Z  sf . -n. -MISC. .   Inspection  of  the  service  bureau  algorithms 
±       i  l     i 

and  the  SMF  data  indicated  that  the  first  two  error  terms,  which  are  due 
to  the  assigning  of  jobs  to  discrete  core  breakpoints,  would  have  a 
minimal  impact  on  total  job  charges. 

Recall  the  four  major  assumptions  of  the  synthetic  approach: 

1.  MISC  charges  have  a  secondary  effect  on  total  job  charges. 

2.  Service  bureau  charging  algorithms  have  the  general  form: 


$  =  Wn -CPU  +  Wo-I0  +  W„ -CPU-Core  +  W. -IO-Core  +  WC«MISC. 
12       3  4  5 


3.   The  assigning  of  jobs  to  discrete  core  breakpoints  has  only  a 
minor  effect  on  total  job  charges. 
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4.   A  synthetic  job  can  be  constructed  to  place  a  prescribed  load 
on  the  CPU,  10  channels,  and  core. 
Assumption  2  appears  valid,  as  has  been  already  noted,  for  many  service 
bureau  algorithms .   The  impact  of  minimum  charges  and  the  assumption  of 
equal  weighting  for  tape  and  disk  activity  in  the  algorithms  are  addressed 
in  Section  V,  Results,  as  is  the  validity  of  Assumption  1.   As  has  been 
noted,  Assumption  3  appears  valid  from  inspection  of  the  service  bureau 
algorithms  and  the  SMF  data  base.   The  validity  of  Assumption  4  is 
discussed  on  p.  19. 

It  is  interesting  to  note  that  the  approach  which  was  finally 
arrived  at  closely  resembled  that  suggested  by  Crowding  [5]  (in  the 
sense  that  an  "average  job"  within  each  core  group  is  computed),  although 
both  approaches  were  formulated  independently  of  each  other.   In  addition, 
both  had  completely  different  purposes,  as  noted  earlier. 

C.   Procedures 

The  following  is  a  list  of  the  steps  required  to  generate  a 
synthetic  mix  for  the  purposes  of  this  study: 

1.  Choose  an  appropriate  synthetic  program. 

2.  Calibrate  the  synthetic  on  the  base  machine  over  a  wide  range 
of  input  parameter  values. 

3.  Determine  a  functional  relationship  between  the  synthetic  input 
parameters  and  CPU  time  and  EXCP  counts. 

4.  Partition  the  workload  into  core  groups,  each  identified  by  the 
mid-point  of  a  40K  increment. 

5.  For  each  core  group,  determine  the  "average"  job — i.e.,  the 
average  CPU  time  and  EXCP  count. 

6.  Scale  these  averages,  if  need  be,  to  reduce  the  CPU  time  to 
10  seconds  of  base  machine  time. 

7.  Generate  a  synthetic  job  which  runs  in  the  prescribed  core  size 
and  which  duplicates  the  scaled  CPU  time  and  EXCP  averages. 

8.  Combine  the  jobs  from  each  core  group  to  form  the  synthetic  mix. 
Four  considerations  should  be  taken  into  account  when  choosing  the 

synthetic  program.   First,  the  program  should  be  capable  of  exercising 
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the  CPU  and  10  in  a  prescribed  manner  via  appropriate  input  parameters. 
Second,  the  program  should  be  portable  across  a  family  of  operating 
systems.   Third,  the  program  should  be  easy  to  instrument  in  order  that 
its  CPU  time  and  10  activity  (EXCP  counts)  may  be  determined  during  the 
calibration  phase.   Finally,  the  program's  source  code  should  not  be 
subject  to  varying  degrees  of  optimization  at  each  different  site.   A 
slightly  modified  version  of  the  Fortran  synthetic  program  described  by 
Shetler  and  Bell  [12]  and  originally  developed  by  Buchholz  [4]  meets  all 
of  these  criteria  and  was  therefore  chosen  as  the  synthetic  program  for 
this  study.   Appendix  A  contains  a  source  listing  of  the  program.   Note 
that  this  program  currently  models  all  10  channel  activity  via  disk  10. 

In  order  to  calibrate  the  synthetic  with  respect  to  CPU  time,  it 
was  first  instrumented  with  STIMER  and  TTIMER  calls  [13] .   A  call  via 
the  STIMER  macro  initiated  an  interval  timer  which  was  thenceforth 
enabled  only  during  the  active  execution  of  the  program.   Succeeding 
calls  via  the  TTIMER  macro  allowed  for  accurate  timings  of  various 
portions  of  the  synthetic  job.   A  wide  range  of  NMAS  (number  of  master 
records)  and  NCPURP  (number  of  CPU  loop  repetitions  per  detail  record) 
values  were  input  and  the  corresponding  CPU  times  on  DMC's  370/165  were 
noted,  as  shown  in  Table  1.   The  NDET  (number  of  detail  records)  and 
NPASS  (number  of  passes)  parameters  were  held  constant  at  12  and  1, 
respectively.   Using  the  model: 

CPU  time  =  X  -NMAS  +  X  -NCPURP  +  X 

linear  regression  analysis  of  the  data  in  Table  1  produced  the  following 
relationship  between  CPU  time  and  the  NMAS  and  NCPURP  parameters : 

(3)  CPU  time  =  .005276-NMAS  +  .00083 2 -NCPURP  +  .9763375 

Simple  source-code  analysis  of  the  program  produced  the  following  rela- 
tionship between  EXCP  counts  and  NMAS  values: 

(4)  EXCP's  =  7 -NMAS  +  58 
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This  count  includes  12  EXCP's  for  the  synthetic's  SYSIN  and  SYSOUT 
activity.   Furthermore,  the  following  DCB  parameter  settings  for  the 
four  temporary  DASD  data  sets  used  by  the  program  are  assumed: 


Unit 

BLKSIZE 

LRECL 

RECFM 

7 

80 

76 

VS 

8 

800 

796 

VS 

9 

800 

796 

VS 

10 

800 

796 

VS 

From  the  inverse  of  (3)  and  (4),  the  proper  values  of  NMAS  and  NCPURP 
needed  for  the  synthetic  to  duplicate  a  prescribed  CPU  time  and  EXCP 
count  can  be  determined: 

(5)  NMAS  =  (EXCP  -  58) /7 

(6)  NCPURP  =  (CPU  -  .9763375  -  ,005276-NMAS) /. 000832 

The  type  4  (step  termination)  SMF  records  were  partitioned  into  the 
four  classes:   H,  D,  J,  OTHER.   The  CPU  time,  total  tape  and  disk  EXCP 
count,  and  core-requested  fields  for  each  type  4  record  were  stripped 
off  and  written  to  one  of  four  files ,  corresponding  to  the  appropriate 
job  class  in  which  the  step  was  executed.   Because  at  the  time  of  data 
collection  the  DMC  configuration  consisted  of  both  a  370/155  and  a  370/ 
165,  it  was  necessary  to  convert  all  CPU  time  to  a  single  base.   There- 
fore, all  370/155  CPU  time  was  expressed  in  equivalent  370/165  time. 
The  data  within  each  of  the  four  classes  was  then  partitioned,  for 
convenience,  into  the  following  nine  core  groups: 
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Core  Group 

Midpoint 

Range 

1 

20K 

0-  40K 

2 

6  OK 

40-  80K 

3 

100K 

80-120K 

4 

140K 

120- 16 OK 

5 

180K 

160-200K 

6 

220K 

200-240K 

7 

260K 

240-280K 

8 

300K 

280-320K 

9 

340K 

320-360K 

An  average  CPU  time  and  EXCP  count  was  then  found  for  each  core 
group  within  each  class.   Several  of  the  core  groups  required  the 
averages  to  be  scaled  in  order  to  reduce  the  average  CPU  time  to  10 
seconds,  a  reasonably  manageable  number.   Table  2  lists  the  number  of 
jobs  in  each  core  group  and  the  total  and  average  CPU  times  and  EXCP 
counts.   Table  3  lists  the  scaled  times  and  counts  with  appropriate 
expansion  factors.   These  factors  were  applied  to  the  cost  of  each 
synthetic  in  order  to  arrive  at  an  estimated  cost  of  running  the  total 
workload.   These  projected  estimates  are  discussed  in  Section  V,  Results. 

From  the  scaled  CPU  times  and  EXCP  counts  in  Table  3,  the  appropri- 
ate NMAS  and  NCPURP  synthetic  input  parameters  were  calculated  using 
equations  (5)  and  (6).   Table  4  lists  the  synthetic  input  parameters 
and  the  REGION  size  required  to  represent  each  core  group.   NMAS  must 
be  forced  to  at  least  a  size  of  12  for  the  synthetic  to  execute  properly. 
It  is  to  be  noted  that  the  core  group  with  midpoint  20K  was  actually 
represented  by  a  synthetic  with  REGION  =  44K.   This  was  required  since 
the  synthetic  itself  was  of  size  44K.   In  any  case,  most  of  the  service 
bureaus  had  a  minimum  core  size  of  from  60K  to  85K  words. 

The  final  collection  of  34  jobs  presented  in  Table  4  constitutes 
the  synthetic  mix  which  was  run  at  DMC  and  at  each  of  the  six  service 
bureaus.   The  following  steps  were  required  to  estimate  the  total  work- 
load cost  at  each  service  bureau: 

1.   Run  the  mix  at  the  priority  which  results  in  normalized  job 
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costs  i.e.,  the  priority  with  a  corresponding  service  factor 
of  1. 

2.  Apply  the  appropriate  expansion  factor  to  the  cost  of  each  job. 
This  represents  an  estimate  of  the  core  group  cost. 

3.  For  each  class,  sum  the  core  group  estimates.   This  sum  then 
represents  the  total  cost  of  each  job  class. 

A  discussion  of  the  results  obtained  from  running  the  generated  mix 
under  these  conditions  now  follows. 

The  synthetic  job  mix  was  run  at  each  of  the  six  service  bureaus 
used  in  the  algorithm  approach  as  well  as  at  the  DMC  itself.   Because 
only  individual  job  costs  were  of  interest  and  not  total  mix  thruput  or 
turnaround  times,  the  particular  sequence  of  jobs  in  the  mix  was  not 
important.   Table  5  lists  the  expected  CPU  times  and  EXCP  counts  for 
class  H  versus  the  actual  times  and  counts  achieved  on  the  DMC  370/165. 
This  data  supports  the  assumption  that  the  synthetic  can  closely  dupli- 
cate specified  CPU  times  and  EXCP  counts.   Tables  6-9  list  the  individual 
job  costs  for  each  core  group  at  DMC  and  each  service  bureau.   In  order 
to  maintain  the  anonymity  of  the  service  bureaus  and  DMC,  no  site  will 
be  identified  by  name  in  Tables  6-10. 


V.   RESULTS 

In  order  to  obtain  the  total  projected  cost  for  each  class,  the 
expansion  factors  listed  in  Table  3  were  applied  to  the  individual 
synthetic  job  costs  (Tables  6-9).   Table  10  contains  these  projected 
class  totals.   Note,  these  are  undiscounted  estimates  and  do  not  include 
unit  record  charges. 

Due  to  the  earlier  assumption  concerning  the  general  form  of  the 
charging  algorithm,  it  was  expected  that  there  would  be  a  discrepancy 
between  the  actual  workload  costs  as  determined  by  the  algorithm 
approach  and  the  projected  costs  using  the  synthetic  approach.   What  was 
not  known  a  priori  was  the  magnitude  of  these  MISC  charges.   As  shown  in 
Table  10  the  relative  error  introduced  into  the  synthetic  approach  by 
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ignoring  MISC  charges  ranged  from  a  low  of  6.3%  for  site  #4  to  a  high 
of  32.1%  for  site  #6.   Because  a  detailed  description  of  the  site 
algorithms  might  compromise  their  confidentiality,  only  general  comments 
will  be  made  concerning  the  effect  of  ignoring  these  MISC  charges. 

For  sites  #1,  #4,  #5,  and  #7  a  large  portion  of  the  MISC  charges  is 
due  to  device  mounts  and  minimum  charges  which  the  "actual  workload  costs" 
include  but  for  which  the  synthetic  approach  did  not  account.   For  these 
sites  the  inaccuracy  due  to  ignoring  MISC  charges  ranged  from  6.3%  to 
13.1%.   For  sites  #2,  #3,  and  #6  which  show  the  higher  percent  difference 
rates,  most  of  the  MISC  charges  are  due  to  charges  for  temporary  work 
space  and  different  charging  factors  for  tape  and  disk  EXCP's  (recall 
that  the  synthetic  approach  did  not  distinguish  between  tape  and  disk 
EXCP's). 


VI.   CONCLUSIONS 

The  synthetic  benchmark  approach  described  here  was  used  to  project 
direct  batch  processing  charges  at  a  number  of  service  bureaus.   The 
approach  relied  on  a  number  of  assumptions  all  of  which  proved  to  be 
valid,  or  at  least  resulted  in  explainable  differences  between  actual 
and  projected  charges. 

Because  it  was  not  possible  to  associate  DMC  job-step  SMF  records 
with  their  corresponding  HASP  initiation  records,  individual  job  turn- 
around times  and  in-queue  times  could  not  be  obtained.   Therefore  no 
attempt  was  made  in  either  the  algorithm  approach  or  in  the  synthetic 
approach  to  account  for  the  impact  of  service  bureau  priority  schemes, 
which  usually  result  in  a  discount  for  deferred  processing  and  a  premium 
for  express  processing. 

Refinement  of  the  synthetic  approach  to  include  temporary  work  space 
and  a  distinction  between  tape  and  disk  activity  should  result  in  a  more 
accurate  estimate  of  projected  workload  costs  for  service  bureaus  whose 
charges  rely  heavily  on  these  measures.   Even  in  its  present  form, 
however,  the  synthetic  approach  described  here  provides  a  reasonably 
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accurate  estimate  (within  13.1%)  of  workload  costs  for  a  given  class  of 
charging  algorithms  within  the  same  vendor  mainframe  family. 
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APPENDIX  A 
SYNTHETIC  PROGRAM  LISTING 
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c 

C  FROM  IBM  SYSTEMS  JOURNAL*  VOL.  8,  NO.  4  BY  BUCHOLZ 

C  TRANSLATED  TO  FORTRAN  BY  J.  F.  MARANZANO»  BELL  TEL.  LASS 

C  MODIFIED  BY  t.  E.  BELL»~  THE  RAND  CORP. 
C 
C 

C  THIS  PROGRAM  IS  FOR  USE  IN  GENERATING  A  SYNTHETIC  PATCH  STREAM. 

C  PARAMETERS  ON  INPUT  CARDS  CAUSE  THE  JOB  TO  USE  THE  COMPUTER 

C  SYSTEM  LIKE  A  CPU-BOUND  JOB.   N  I/O  BOUND  JOB.  OR  *    JOB  WITH 
C       "  ANY  DEGREE  OF  CPU  AND  I/O  MIX.   EACH  PARAMETER  CARn  CAUSES 

C  ONE  OR  MORE  TIMED  PASSES  THROUGH  THE  SPECIFIED  PASs.   AT  THE 

C  END  OF  WORK  FOR  ONE  PARAMETER  CARD.  THE  RESULTS  ARf  PRINTED 

C  SEVERAL  PARAMETER  CARDS.   END  OF  INPUT  IS  INDICATED  BY  A 

C  OUT.   DIFFERENT  TYPES  OF  PASSES  CAN  BE  SPECIFIED  By  INPUTTING 
_C  _     _  DUMMY  PARAMETER  CARD  WITH  NPASSfO. 

C  THE  JOB  FIRST  INITIALIZES  FILES  FOR  A  SET  OF  PASSED  (SPECIFIED 

C  BY  ONE  PARAMETER  CARD).   EACH  PASS  HAS  ITS  OWN  INITIALIZATION 

C  AND  TERMINATION  TO  REWIND  FILES  AND  TO  SUMMARIZE  RESULTS. 

C  HOWEVER.  OUTPUT  IS  HELD  UNTIL  ALL  PASSES  OF  THE  PApAMETER  CARD 

C  ARE  COMPLETED.   THIS  ELIMINATES  SY_SOUT  BEING  OVERLAPPED  WITH 

C        "TESTS. 

C 

C  I/O  CAN  BE  DIRECTED  TO  TAPE.  DISC.  OR  DRUM.   READING  AND 

C  WRITING  ARE  UNFORMATTED  TO  REDUCE  FORTRAN  CONVERSION  ROUTINE 

C  EXECUTION.   I/O  UNITS  ARE  USED  AS  FOLLOWS  - 

C  1.   MASTER  FILE  WRITTEN  DURING  PASS  SET  INITIALIZATION.   IT  IS 

C  SUBSEQUENTLY  READ  FOR  WRITING  TO  FILE  4. 

C  2,   DETAIL  FILE  WRITTEN  DURING  PASS  SET  INITIALIZATION.   IT  IS 

C  SUBSEQUENTLY  READ  FOR  WRITING  TO  FILE  3. 

C  3.   DETAIL  FILE  WRITTEN  DURING  PASS  FROM  FILE  2. 

C  4.   MASTER  FILE  WRITTEN  DURING  PASS  FROM  FILE  1. 

C_ 

C  IF  A  PASS  HAS  MASTER  RECORDS.  ONE  OR  MORE  MASTER  RECORDS  ARE 

C  PROCESSED  FOR  EACH  DETAIL  RECORD.   EACH  DETAIL  RECoRD  PROCESSED 

C  IS  FOLLOWED  BY  THE  NUMBER  OF  CPU  REPETITIONS  INDICATED  ON  THE 

C  PARAMETER  CARD.   IF  NUET  =  0,  NO  CPU  REPETITIONS  WtLL  BE 

C  PERFORMED.   HOWEVER.  IF  NcPURP  =  0.  OETAIL  RECORDS  WILL  BE 

C  PROCESSED  IF  REQUESTED. 

C 

C  IF  A  PASS  HAS  NO  MASTER  RECORDS*  NCUPRP  REPETITIONS  OF  THE 

C  CPU  LOOP  WILL  BE  PERFORMED. 

C 

C 

C  THE  PROGRAM  IS  WRITTEN  IN_ FORTRAN,  AND  CALLS  A  ROUTINE  NAMED 

C  ZTIME  TO  OBTAIN  THE  CURRENT  COMPUTER  CLOCK  VALUE.   THIS 

C  ROUTINE  MUST  BE  WRITTEN  FOR  EACH  SYSTEM. 

C 

C 

C  INPUT 

C  NMAS       NUMBER  OF  MASTER  RECORDS.   NMAS  MUsT  BE  GREATER 

C  '     THAN  ZERO  FOR "ANY  T/O  TO  BE  DONE. 

C  NOET      NUMBER  OF  DETAIL  RECORDS  (LESS  THAN  OR  EQUAL  TO 

C  NMAS) 

C  NCPURP     NUMBER  OF  REPETITIONS  OF  CPU  MINOR  LOOP  FOR 
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C  EACH  DETAIL  RECORD .   IF  NO  DETAIi  OR  MASTER 

C  RECORDS,  NCPUR  IS  THE  TOTAL  NUM&rR  OF 

C  REPETITIONS  PtR  PASS  IN  TOTALLY  cPU-BOUNO 

C  PASSES. 

C  NPASS      NUMBER  OF  IDENTICAL  PASSES  THROUGH  LOOP  DEFINED 

C  BY  MMASS*  NDET,  AND  NCPURP 

C 

C  THE  FORMAT  OF  THE  PARAMETFR  INPUT  CARD  IS  AS  SHOWN  BELOW. 

C  THE  ZEROS  WOULD.  OF  COURSE*  BE  REPLACED  WITH  THE  DfSIRED 

C  VALUES.   THE  VARIABLE  IDENTIFIERS  ARE  PRESENT  ONLY  FOR 

C  CONVENIENCE.   (PARAMETER  CARDS  ARE  READ  UNDER  FORMAT  88.) 

C 

C23456 7 890 1234567890 123456 7890 1234567890 123466 7890 123456789M 234567890 1234567890 

CNMAS=000000(),NDET=0000000,NCPURP=0000000,NRASS=00 

C 

C 

c 

C  THE  MASTER  FILE  AND  DETAIL  FILE  ARE  DEFINED  HERE 

C  BY  THE  LENGTH  OF  THE  'RECORD'  VARIABLE  WHICH  WILL  °E  WRITTEN, 

C  'RECORD'  MUST  BE  DIMENSIONED  AS  RECORD(N)  WITH  N  1  OR  MORE. 

0001  INTEGER  RECORD  (47) 

0002  INTEGER  START,  SUM,  TABLE(IOOO) 

0003  INTEGER  XI ,X2, X3,X4,X5, X6 

0004  DIMENSION  TIMESU00.2) 

0005  DATA  N/10/, START/100/ 
C 

0006  CALL  BTIME 

0007  WRITE(6,87) 
C 

C  READ  INPUT  PARAMETERS 
C 

0008  1000   CALL  ZTIME  (TIMEN)  *»»««*««■ 

0009  READ  (5,88)  NMAS,  NDET ,.  N_CPURP_,NP_ASS  « 
C  FORCE  NMAS  TO  AT  LEAST  12 

0010  IF  (NMAS. LT. 12)  NMAS=12 

0011  wRITE(6»b9)  NMAS,  NDET,  NCPURP,  NPASS  » 
C  « 
C  CHECK  FOR  UNUSUAL  INPUT  » 
C  » 
C  END  OF  INPUT  CARDS                                                       * 

0012  IF  (NPASS  .LE.  0)  CALL  EXIT  » 
C,  MORE  DETAIL  THAN  MASTER   —   DISREGARD                                • 

0013  IFINMAS  .GE.  NDET)  GO  TO  30  • 

0014  WR1TE(6.90)  * 

0015  GO  TO  1000  .  .  ._„  ...........                                 * 

C  REQUESTED  CPU  REPETITIONS,  BUT  NDET  =  0,  NMAS  GT  0                     » 

0016  30  IF((NMAS  .LE.  0)  .OR.  (NDET  .GT,  0)  .OR.  (NCPURP  .LE.  0))  GO  TO  32        * 

0017  WRITE(6,91)  • 

0018  32  CONTINUE  • 
C  * 
C  END  OF  PARAMETER^ INPUT  PROCESSING  * 
C  • 
C  INITIALIZE  TABLE  WITH  NUMRERS                                            « 

0019  K  =  N«»3  tt 

0020  DO  200  1=1, K  P    * 
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0021 

200 

TAt)LE(l)  =  START+I-1 

0022 

MKEY  =  0 

0023 

C 

LKEY  =  0 

0024 

c 

IF(NMAS  .EQ.O)  GO  TO  280 

c 

MASTER  GENERATION 

0025 

RECORO(l)  =  0 

0026 

DU  250  MKEY=1»NMAS 

0027 

RSUM  =  0. 

0028 

MCHK  =  MKEY 

0029 

250 

WRITE  (7)  MKEY,  RSUM,  MCHK, 

RECORD 

0030 

RECORO(l)  =  1 

0031 

WRITE  (7)  MKEY,  RSUM,  MCHK, 

RECORD 

0032 

RECOHO(l)  =  0 

0033 

C 

ENOFILE  7 

0034 

C 
251 

IFfNDET  .EQ.  0 ) GO  TO  280 

C 

DETAIL  GENERATION 

0035 

NRATO  =  NMAS/NDET 

0036 

DO  275  LKEY  =  NRATO, NMAS 

,  NRATO 

0037 

OSUM  =0. 

0038 

LCHK  =  LKEY 

0039 

275 

WRITE  (8)  LKEY,  OSUM,  LCHK, 

RECORD 

0040 

RECOHD(l)  =  1 

0041 

WRITE  (8)  LKEY,  OSUM,  LCHK, 

RECORD 

0042 

RECORD) 1)  =  0 

0043 

C 

c 
c 

280 

C 

ENUFILE  8 

0044 

CONTINUE 

c 
c 
c 

c 
c 

DO  LOOP  NPASS  TIMES 

0045 

CO  1  ICNT  =  1,  NPASS 

IF  NO  MASTER  OR  DETAILED 

RECORD 

0046 

IFINMAS  .EQ.  0)  GO  TO  285 

0047 

KCNT  =  0 

0048 

REWIND  10 

0049 

REWIND  7 

0050 

(EAD  (7)  MKEY,  RSUM,  MCHK,  1 

RECORD 

0051 

IF(NDET  ,GT.  0)  GO  TO  283 

0052 

LKEY  =  999999 

0053 

GO  TO  285 

0054 

283 

REWIND  9 

0055 

REWIND  8 

0056 

C 

READ  (8)  LKEY,  OSUM,  LCHK, J 

RECORD 

C 
C 
C 

START  TIMED  PASS 

«• 

A  S 

S  E 

S  T 

U 

S  P 

E 
T 


«»«»»»«« 


mnnnnmv 


p  S 
A  E 
S    T 

s  u 
p 


»«»«»««« 


<nnnnnnnnnnnnnnnnnnntj><nnnnnnnnnn»»»««»»»«» 
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0057 

285 

0058 

C 

c 

0059 

400 

c 

0060 

425 

0061 

c 

0062 

450 

C 

C 

c 

c 

c 

c 

c 

c 

0063 

0064 

0065 

0066 

0067 

0068 

0069 

0070 

350 

C  . 

0071 

0072 

0073 

0074 

0075 

300 

C 

C 

C 

0076 

301 

C 

C 

C 

C 

0077 

460 

C 

0078 

0079 

0080 

0081 

0082 

0083 

0084 

C 

C 

C 

0085 

475 

0086 

0087 

CALL    ZTIME(FSTRTM) 
TlMESdCNT,     1)     =    FSTRTM 


IF (MKEY-LKEY)       475.450,425 

SEQUENCE    ERROR 
wRITE(6,94) 

STOP         6 

IF  NOT  REQUESTED,  SKI_P  TH£  COMPUTE  KERNEL 
IFINCPURP  ,LE.  0)  GO  TO  301 


COMPUTE  KERNEL 

REPEAT  NCPUR  TIMES  FOR  EACH  DETAIL  RECORD  (OR  JUST  NCPUR  IF 
NMAS  =  NDET  =  0) 


START  CPU  LOOP 
00  300  1=1,  NCPURP 
SUM  =  0 
IU  =  _0 
J  =  0 

DO  350  K=1,N 
J  =  J* (6»IU*1) 
SUM  =  SUM  ♦  TABLE (J) 

iu  =iu  *k 

LSUM  =  (N*r(Nii~)  >/2 

IF(START  .EO.  <SUM-LSUM«LSUM)/N*1)60  TO  300 
wRITE(6,96) 

STOP   5 

CONTINUE 

END  OF  CPU  LOOP  . 

IF  NO  I/O  REQUESTED.  SKIP  I/O  StCTION 
IFINMAS  .EQ.  0)  GO  TO  60(J 


START  OF  I/O  SECTION 

CONTINUE 

WRITE  DETAIL  RECORDS 

RSUM  =  SUM 

DSUM  =  SUM 

ICHK  =  LCHK 

KCNT  =  KCNIT+1 
aRITE  (9)  LKEY,  DSUM,  LCHK,  RECORD 
READ  (8)  LKEY,  DSUM,  LCHK,  RECORD 
IF(RECORDd)  .EQ.  1)  GO  TO  500 

WRJTE  MASTER _ 

WRITE  (10)  MKEY,  RSUM,  MCHK,  RECORD 
READ  (7)  MKEY,  RSUM,  MCHK,  RECORD 
IF(RECORDd)  .EQ.  1)  GO  TO  600 


tt<nnnnnnt»<nnnnnnnnnnnt 


CPU  LOOP 


,904«4»»(HmH>»»IU.IHU 


!.**»*•»**«»»»»**•»»»» 


I/O  LOOP 
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0086 


0089 
0090 


0091 

0092 
0093 


0094 
0095 
0096 


0097 


0098 
0099 
0100 


0101 
0102 
0103 
0104 
0105 
0106 
0107 
0108 
0109 
0110 
0111 
0112 
0113 
0114 


C 

c 

500 
C 

c 

c 

600' 
C 


700 
C 
C 
1 

c 
c 
c 
c 

c 
c 
c 


GO  TO  400 

NO  DETAILED  RECORDS  LEFT 

LKEY  =  999999 
GO  TO  475 

NO  MASTER  RECORDS  LEFT 

~  CONTINUE 
GET  END  TIME 
CALL  ZTIME (FENUTM) 
TIMES  UCNT.  2)  =  FENDTM 

END  OF  I /O  SECTION 

END  OF "TIMED  PASS" 


«.«»«««««»»»«»«««««««« 


««»««»«««»««»«««««»«««•«#»•«»««««•»»««»*« 

«««««««» 


IF  I/O  SECTION  HAS  BEEN  USED,  CHECK  FOR  CORRECT  OPrRATION 
I F  t  ( NM AS. EQ .  0 )  .  OR  ._  ( NDET . EQ^Oi  )  _GO  TO  1_ 
IF  '<  ICHK-  (KCNT»NRATO)  ) 700,1 t700 
wRITE<6,93)  ICHK,ICNT,NRATO 


CONTINUE 


PUT  OUT  TOTAL  ELAPSED  TIME  FOR  THIS  SET  OF  PASSES 


1100 


CALL  ZTIME(TIMEF) 
CIFF  =  TIMEF  -  TIMEN 
l*RITE(6,95)  TIMEN,  TIMEF,  DIFF 


COMPUTE  RESULTS 

wRITE<6»97) 

STIMES  =  0. 

SSTIME  =  0. 

CO  1100  ICNT  =  1 ,  MPASS 

TIMEON  =  TlMESdCNT,  1) 

TIMEOF  =  TlMESdCNT,  2) 

DIFF  =  TIMEOF  -  TIMEON 

itiRITE  (6,99)  TIMEON,  TIMEOF,  DIFF 

STIMES  =  STIMES  ♦  DIFF 

SSTIME  =  SSTIME  "♦  DIFF  #*  2 

CONTINUE 

TMEAN  =  STIMES  /  TlOAT "(NPASS'ji  "' 

TSTDEV  =  SORT ((SSTIME  /  FLOAT (NPASS) )  -  TMEAN  »»  2) 

i»RITE<6,98)  TMEAN,  TSTOEV 


P  C  » 
A_  H  _« 

S  E  • 
S  C  • 

K  » 


R  * 

F  E  • 
I  S  « 


N  JJ  « 

A  L  • 

L  T  « 

S  * 


«»«»*»«« 
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C         GO  GET  NEXT  CONTROL  CARD  » 

C  « 

0115  REWIND  7  » 

0116  REWIND  8  « 

0117  REWIND  9  « 

0118  REWIND  10  » 

0119  GO  TO  1000  « 
C  •»»«»■»«»« 
C  FORMAT  STATEMENTS 

C 

0120  87  FORMATI24H1  SYNTHETIC  BATCH  STREAM) 

0121  88  FORMAT(6X,  17,  6X,  17.  8X,  I7»  7X.  12) 

0122  89  FORMAT (10HO  MASTER  = ♦ 18, 4X » BhDET A IL  =  » 18 , 4X , 1 7HCPU  REPETITIONS  =, 

1    IB»4X,8HPASSES  =  ,15) 

0123  90  FORMAT  (38H   MOtiF  DETAILS  THAN  MASTERS  SPECIFIED^ 

0124  91  F0RMAT(30H   NO  CPU  LOOPS  SINCE  NDE 1  =  0.) 

0125  93  FORMaT(23H  CHECKSUM  ERROR,  ICHK=  ,i6,7H  KCNT=  ,16, 8H  ikATO=  ,  1 6 ) 

0126  9~4  FORMAT  (15H  SEQUENCE  ERROR) 

U127         95  FORMAT (10HOON-TIME  = , F 1 0 . 3 »4x , 1 0HOFF- Tl ME  =,F10.3. 

1    4X,20HTOTAL  ELAPSED  TIME  s,F10.J) 

0128         96  FORMAT  (14m  r.DMPUTF  ERROR) 

0129'        v7  FORMAT(lH0,llX,5riSTAprT,llX»3HENO,I^X,12HELAPSED  TIME) 

0130  96  FORmaTUh  ,22X,  19HMEAN  ELAPSED  TIMl  =,Flu,3, 

1    22H   STANDARD  DEVIATION  =,F10.3//) 

0131  99  FORMAT (1H  ,  8X,  F10.3,   6X,  Fl0,3,   7X,  F10.3) 

0132  END 
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0001  SUBROUTINE  ZTIME  (TIMET) 
C 

C  RETURNS  CPU  ELAPSED  TIME_LN  SECONDS 

C  USED  ONLY  DURING  THE  CALIBRATION  PHASE  —  HENCE  A  DUMmY  ITIME 

C  FOR  THE  RUNNING  VERSION  OF  THE  SYNTHETJC 

C 

0002  0UMMY=0,0 

C  ITIME  RETURNS  NO.  OF  TIMER  UNITS  REMAINING  (1  TIMER  UmIT=26  USEC) 

0  0  03  1  =  1 TIME  (DUMMY). 

C  STIMER  SETTING  WAS  16**4/1UU 

0004  TIMET= 16, 0**4/ 100. 0-26,0* (I*. 000001) 

0005  RETURN 

0006  END 


0001  SUBROUTINE  BTIME 
C  DUMMY  BTIME 

0002  1=1 

0003  RETURN 

0004  END 


0001  FUNCTION  ITIME  (DUMMY) 
C  DUMMY  ITIME 

0002  ITIME=0 

0003  RETURni 

0004  END 
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APPENDIX  B 
TABLES 
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NMAS 


TABLE  1 
SYNTHETIC  PROGRAM  CALIBRATION 
(NDET  =  12,  NPASS  =  1) 


NCPURP 


DMC  370/165 
Time  (sec) 


NMAS 


NCPURP 


DMC  370/165 
Time  (sec) 


12 

0 

.765 

12 

4000 

3.757 

50 

0 

1.005 

50 

4000 

3.827 

100 

0 

1.201 

100 

4000 

4.456 

500 

0 

3.541 

500 

4000 

5.721 

900 

0 

5.837 

900 

4000 

8.337 

1000 

0 

6.503 

1000 

4000 

8.939 

5000 

0 

32.239 

5000 

4000 

32.764 

9000 

0 

50.256 

9000 

4000 

50.199 

1000.0 

0 

53.161 

10000 

4000 

53.035 

12 

500 

1.015 

12 

16000 

14.530 

50 

500 

1.201 

50 

16000 

15.372 

100 

500 

1.561 

100 

16000 

14.873 

500 

500 

3.401 

500 

16000 

17.801 

900 

500 

5.491 

900 

16000 

19.682 

1000 

500 

5.987 

1000 

16000 

20.204 

5000 

500 

28.644 

5000 

16000 

41.703 

9000 

500 

50.173 

10000 

500 

53.986 

12 

1000 

1.528 

12 

90000 

75.672 

50 

1000 

1.787 

50 

90000 

76.574 

100 

1000 

1.937 

100 

90000 

76.231 

500 

1000 

4.247 

500 

90000 

79.639 

900 

1000 

7.319 

900 

90000 

79.153 

1000 

1000 

7.105 

5000 

1000 

29.586 

9000 

1000 

51.677 

10000 

1000 

50.915 
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TABLI 
CORE  GROUP 

I  2 
AVERAGES 

No.  of 

Total  165  CPU 

Total 

Average 

Average 

Class   Group 

Jobs 

Time  (sec) 

EXCP  Count 

CPU  Time 

EXCP  Count 

H      1. 

5538 

5538 

554100 

1 

100 

2. 

5819 

35919 

22452704 

6 

3858 

3. 

2970 

61066 

23538768 

20 

7925 

4. 

619 

30351 

9760500 

49 

15768 

5. 

907 

32452 

10649700 

35 

11741 

6. 

175 

10000 

1474500 

57 

8425 

7. 

57 

18912 

1628200 

331 

28564 

8. 

13 

372 

112600 

28 

8661  - 

9. 

26 

4698 

1514400 

180 

58246 

D      1. 

5006 

5254 

701600 

1 

140 

2. 

10103 

16254 

4933400 

1 

488 

3. 

12253 

33275 

6731700 

2 

549 

4. 

3169 

17912 

1766100 

5 

557 

5. 

1958 

9156 

1502000 

4 

767 

6. 

154 

635 

94200 

4 

611 

7. 

11 

344 

2200 

31 

200 

8. 

31 

312 

9800 

10 

316 

9. 

4 

340 

20000 

85 

5000 

J      1. 

5536 

6203 

744000 

1 

134 

2. 

6782 

21810 

10295300 

3 

1518 

3. 

8014 

49324 

13781100 

6 

1719 

4. 

1362 

12035 

2118600 

8 

1555 

5. 

799 

17597 

2977600 

22 

3726 

6. 

51 

866 

64400 

16 

1262 

7. 

16 

166 

7300 

10 

456 

8. 

1 

6 

100 

6 

100 

OTHER    1. 

2209 

2627 

485500 

1 

219 

2. 

6562 

33253 

20882448 

5 

3182 

3. 

4071 

67709 

22316256 

16 

5481 

4. 

1270 

38334 

6530300 

30 

5141 

5. 

652 

65851 

6040800 

100 

9265 

6. 

205 

29671 

1483700 

144 

7237 

7. 

43 

19431 

503400 

451 

11706 

8. 

6 

6 

800 

1 

133 

34 


TABLE  3 

CORE  GROUP  SCALED  AVERAGES  AND 
EXPANSION  FACTORS 


Core 

Scaled  Avg. 

Scaled  Avg. 

Expansion 

Class 

Group 

CPU  Time  (sec) 

EXCP  Count 

Factor 

H 

1. 

1 

100 

5538 

2. 

6 

3858 

5819 

3. 

10 

3963 

5940 

4. 

10 

3218 

3033.1 

5. 

10 

3355 

3174.5 

6. 

10 

1478 

997.5 

7. 

10 

863 

1886.7 

8. 

10 

3093 

36.4 

9. 

10 

3236 

468 

D 

1. 

1 

140 

5006 

2. 

1 

488 

10103 

3. 

2 

549 

12253 

4. 

5 

557 

3169 

5. 

4 

767 

1958 

6. 

4 

611 

154 

7. 

10 

65 

34.1 

8. 

10 

316 

31 

9. 

10 

588 

34 

J 

1. 

1 

134 

5536 

2. 

3 

1518 

6782 

3. 

6 

1719 

8014 

4. 

8 

1555 

1362 

5. 

10 

1694 

1757.8 

6. 

10 

789 

81.6 

7. 

10 

456 

16 

8. 

6 

100 

1 

OTHER 

1. 

1 

219 

2209 

2. 

5 

3182 

6562 

3. 

10 

3426 

6513.6 

4. 

10 

1714 

3810 

5. 

10 

927 

6520 

6. 

10 

503 

2952 

7. 

10 

260 

1939.3 

8. 

1 

133 

6 
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TABLE 

4 

SYNTHETIC 

MIX 

SPECIFICATION 

Core 

Synthetic  Input 

Parameters 

(NDET=12 

,NPASS=1) 

REGION 

Class 

Group 
1. 

NMAS 

NCPURP 

Size  (K) 

H 

12 

0 

44 

2. 

543 

2595 

60 

3. 

558 

7307 

100 

4. 

451 

7986 

140 

5. 

471 

7859 

180 

6. 

203 

9558 

220 

7. 

115 

10116 

260 

8. 

434 

8094 

300. 

9. 

454 

7967 

340 

D 

1. 
2. 
3. 
4. 
5. 
6. 
7. 
8. 
9. 

12 
61 
70 
71 
101 
79 
12 
37 
76 

0 

0 

786 

4386 

2994 

3133 

10839 

10611 

10364 

44 
60 
100 
140 
180 
220 
260 
300 
340 

J 

1. 
2. 
3. 
4. 
5. 
6. 
7. 
8. 

12 
209 
237 
214 
234 
104 
57 
12 

0 
1107 
4535 
7085 
9362 
10186 
10484 
6000 

44 
60 
100 
140 
180 
220 
260 
300 

OTHER 

1. 

2. 
3. 
4. 
5. 
6. 
7. 
8. 

23 

446 

481 

237 

124 

64 

29 

12 

0 

2008 

7796 

9343 

10059 

10440 

10662 

0 

44 
60 
100 
140 
180 
220 
260 
300 

36 


TABLE  5 

EXPECTED  VS.  ACTUAL  CPU  TIMES  AND  EXCP  COUNTS 
FOR  CLASS  H  ON  THE  DMC  370/165 


Core 

Expected 

Expected 

Actual 

Actual 

Groups 

CPU  Time 

(sec) 

EXCP's 

CPU  Time  (sec) 

EXCP's 

1. 

1 

100 

.89 

142* 

2. 

6 

3858 

4.92 

3859 

3. 

10 

3963 

8.49 

3964 

4. 

10 

3218 

9.16 

3215 

5. 

10 

3355 

9.18 

3355 

6. 

10 

1478 

9.78 

1479 

7. 

10 

863 

9.55 

863 

8. 

10 

3093 

9.59 

3096 

9. 

10 

3236 

9.39 

3236 

*due  to  NMAS  forced  to  12 
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work  at  NBS.  The  magazine  highlights  and  reviews  such 
issues  as  energy  research,  fire  protection,  building  tech- 
nology, metric  conversion,  pollution  abatement,  health 
and  safety,  and  consumer  product  performance.  In  addi- 
tion, it  reports  the  results  of  Bureau  programs  in 
measurement  standards  and  techniques,  properties  of 
matter  and  materials,  engineering  standards  and  serv- 
ices, instrumentation,  and  automatic  data  processing. 

Annual  subscription:  Domestic,  $9.45;  Foreign,  $11.85. 

NONPERIODICALS 

Monographs — Major  contributions  to  the  technical  liter- 
ature on  various  subjects  related  to  the  Bureau's  scien- 
tific and  technical  activities. 

Handbooks — Recommended  codes  of  engineering  and 
industrial  practice  (including  safety  codes)  developed 
in  cooperation  with  interested  industries,  professional 
organizations,  and  regulatory  bodies. 

Special  Publications — Include  proceedings  of  confer- 
ences sponsored  by  NBS,  NBS  annual  reports,  and  other 
special  publications  appropriate  to  this  grouping  such 
as  wall  charts,  pocket  cards,  and  bibliographies. 

Applied  Mathematics  Series — Mathematical  tables, 
manuals,  and  studies  of  special  interest  to  physicists, 
engineers,  chemists,  biologists,  mathematicians,  com- 
puter programmers,  and  others  engaged  in  scientific 
and  technical  work. 

National  Standard  Reference  Data  Series — Provides 
quantitative  data  on  the  physical  and  chemical  proper- 
ties of  materials,  compiled  from  the  world's  literature 
and  critically  evaluated.  Developed  under  a  world-wide 


program  coordinated  by  NBS.  Program  under  authority 
of  National  Standard  Data  Act   (Public  Law  90-396). 

NOTE:  At  present  the  principal  publication  outlet  for 
these  data  is  the  Journal  of  Physical  and  Chemical 
Reference  Data  (JPCRD)  published  quarterly  for  NBS 
by  the  American  Chemical  Society  (ACS)  and  the  Amer- 
ican Institute  of  Physics  (AIP).  Subscriptions,  reprints, 
and  supplements  available  from  ACS,  1155  Sixteenth 
St.  N.  W.,  Wash.  D.  C.  20056. 

Building  Science  Series — Disseminates  technical  infor- 
mation developed  at  the  Bureau  on  building  materials, 
components,  systems,  and  whole  structures.  The  series 
presents  research  results,  test  methods,  and  perform- 
ance criteria  related  to  the  structural  and  environmen- 
tal functions  and  the  durability  and  safety  character- 
istics of  building  elements  and  systems. 

Technical  Notes — Studies  or  reports  which  are  complete 
in  themselves  but  restrictive  in  their  treatment  of  a 
subject.  Analogous  to  monographs  but  not  so  compre- 
hensive in  scope  or  definitive  in  treatment  of  the  sub- 
ject area.  Often  serve  as  a  vehicle  for  final  reports  of 
work  performed  at  NBS  under  the  sponsorship  of  other' 
government  agencies. 

Voluntary  Product  Standards — Developed  under  pro- 
cedures published  by  the  Department  of  Commerce  in 
Part  10,  Title  15,  of  the  Code  of  Federal  Regulations. 
The  purpose  of  the  standards  is  to  establish  nationally 
recognized  requirements  for  products,  and  to  provide 
all  concerned  interests  with  a  basis  for  common  under- 
standing of  the  characteristics  of  the  products.  NBS 
administers  this  program  as  a  supplement  to  the  activi- 
ties of  the  private  sector  standardizing  organizations. 

Federal  Information  Processing  Standards  Publications 
(FIPS  PUBS) — Publications  in  this  series  collectively 
constitute  the  Federal  Information  Processing  Stand- 
ards Register.  Register  serves  as  the  official  source  of 
information  in  the  Federal  Government  regarding  stand- 
ards issued  by  NBS  pursuant  to  the  Federal  Property 
and  Administrative  Services  Act  of  1949  as  amended, 
Public  Law  89-306  (79  Stat.  1127),  and  as  implemented 
by  Executive  Order  11717  (38  FR  12315,  dated  May  11, 
1973)  and  Part  6  of  Title  15  CFR  (Code  of  Federal 
Regulations). 

Consumer  Information  Series — Practical  information, 
based  on  NBS  research  and  experience,  covering  areas 
of  interest  to  the  consumer.  Easily  understandable 
language  and  illustrations  provide  useful  background 
knowledge  for  shopping  in  today's  technological 
marketplace. 

NBS  Interagency  Reports  (NBSIR) — A  special  series  of 
interim  or  final  reports  on  work  performed  by  NBS  for 
outside  sponsors  (both  government  and  non-govern- 
ment). In  general,  initial  distribution  is  handled  by  the 
sponsor;  public  distribution  is  by  the  National  Technical 
Information  Service  (Springfield,  Va.  22161)  in  paper 
copy  or  microfiche  form. 


Order  NBS  publications  (except  NBSIR's  and  Biblio- 
graphic Subscription  Services)  from:  Superintendent  of 
Documents,  Government  Printing  Office,  Washington, 
D.C.  20402. 


BIBLIOGRAPHIC  SUBSCRIPTION  SERVICES 


The  following  current-awareness  and  literature-survey 
bibliographies  are  issued  periodically  by  the  Bureau: 
Cryogenic    Data    Center    Current    Awareness    Service 

A    literature    survey   issued   biweekly.    Annual    sub- 
scription: Domestic,  $20.00;  foreign,  $25.00. 

Liquefied  Natural  Gas.  A  literature  survey  issued  quar- 
terly. Annual  subscription:  $20.00. 

Superconducting   Devices   and   Materials.   A   literature 


survey  issued  quarterly.  Annual  subscription :  $20.00. 
Send  subscription  orders  and  remittances  for  the 
preceding  bibliographic  services  to  National  Bu- 
reau of  Standards,  Cryogenic  Data  Center  (275.02) 
Boulder,  Colorado  80302. 
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